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ABSTRACT 


Conventional  methods  for  comparlntr  the  relative  productivity  of 
schools  employ  least  squares  regression  to  find  expected  achievement  of 
schools  with  the  same  Input  characteristics.  The  result  Is  that  one 
typically  contrasts  the  relative  effects  of  "predictor"  variables  on 
achievement  rather  than  comparing  school  units  with  respect  to  their 
Input/output  efficiency. 

A  non-linear  method  for  comparing  the  efficiency  of  decision 
making  units  (school)  Is  presented  and  Is  applied  to  elementary  schools 
in  an  urban  school  district.  The  method  Is  cross-validated  by  means  of 
discriminant  analysis  and  a  nidlmentary  sensitivity  test  Is  made.  The 
method  Is  found  to  discriminate  between  efficient  and  Inefficient  schools 
and  Is  relatively  robust  under  changes  In  Input  measures. 
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BACKQROUND 


Methods  for  evaluating  the  relative  productivity  of  decision  maklnK 
units  in  the  public  sector  have  lagged  behind  similar  applications  where 
production  functions  were  more  directly  obtainable.  Chamos  and  Cooper 
In  [3l  recently  reviewed  relevant  development  In  economic  theory  from 
the  standpoint  of  managerial  economics.  They  also  described  the  methodological 
developments  undertaken  with  E.  Rhodes  [4,3l  to  measure  the  efficiency  of 
"decision  making  units"  with  special  reference  to  not-for-profit  enterprise 
and  government  agencies.  This  resulted  In  a  technique  that  they  call 
Data  Envelopment  Analysis  for  measuring  and  distinguishing  different  kinds 
of  efficiencies  such  as  "program  efficiency"  and  "managerial  efficiency. " 

The  utility  of  the  theory  has  been  demonstrated  In  the  secondary  analysis  of 
Program  Follow-Through  evaluation  data  I4,10l--an  Important  federally- 
funded  Intervention  aimed  at  Improved  education  for  disadvantaged  children. 

In  the  works  cited,  Chames,  Cooper  and  Rhodes  succeeded  In 
quantifying  the  relative  efficiency  of  decision  making  units  (DMlPs)  within 
a  set  of  like  units  and,  further,  conceptualised  a  method  for  comparing  the 
relative  efficiency  of  two  sets  of  units  classified  on  some  a  priori  basis  14), 
Only  a  limited  set  of  variables  were  selected  for  Illustrative  purp^''ses,  the 
objective  being  to  show  how  one  might  compare  two  sets  of  schools  operating 
under  different  programs. 

The  present  paper  Is  directed  to  a  study  of  the  possible  use  of 
these  measures  by  management  at  the  Individual  school  level.  For  this 
purpose,  a  comprehensive  set  of  variables  will  be  employevl  and  applied 
to  all  schools  In  a  single  district. 
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We  want  to  concentrate  on  the  applicability  of  DEA  In  the  manage¬ 
ment  of  an  urban  public  school  district  with  emphasis  on  (a)  the  Identlfl- 
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cation  of  school  units  which  make  better  use  of  Input  resources  In  terms 
of  measured  outputs,  and  (b)  the  obtaining  of  estimates  of  the  extent 
to  which  Inputs  are  underutilized  In  DMU's  which  are  relatively  unproductive. 

In  addition,  we  shall  contrast  the  results  of  DEA  with  those 
obtained  In  the  more  usual  manner,  such  as  least-squares  regression 
models.  We  shall  also  cross-valldate  the  resulting  classification  of 
DEA  by  means  of  discriminant  analysis.  Finally,  a  rudimentary  sensitivity 
analysis  la  performed  to  test  the  measure-dependency  of  DEA. 

DATA  ENVELOPMENT  ANALYSIS 

In  their  work,  Chames,  Cooper  and  Rhodes  Introduce  first  a  conceptual 
model  which.  In  ratio  form,  makes  It  possible  to  relate  efficiency  measure¬ 
ment  approaches  In  engineering,  economics,  etc.  to  each  other.  After 
exhibiting  this  property  of  the  conceptual  model,  Chames,  Cooper  and  Rhodes 
then  show  how  this  model  may  be  replaced  by  an  equivalent  crxlinary  linear 
programming  problem.  The  latter,  which  has  all  the  power  and  convenience  of 
ordinary  linear  programming  Is  the  one  we  shall  employ,  along  with  the 
associated  definition  of  efficiency  which,  In  economics,  Is  designated  as 
Pareto  Efficiency.*  The  latter,  as  given  in  (3l,  may  be  paraphrased  here  as 
follows: 

"A  DMU  (Decision  Making  Unit)  Is  not  efficient  In  producing  Its 
output  (from  given  sunounts  of  Input)  If  It  can  be  shown  that 
some  redistribution  of  resources  will  result  In  the  same  amount 
of  this  output  with  less  of  some  resource  and  no  more  of  any  other 
resource.  Conversely,  a  firm  Is  efficient  if  this  Is  not  possible." 

« 

Also  called  Pareto-Koopmans  Efficiency.  See  1 3). 
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This  18  the  definition  of  efficiency  we  ahall  employ  with  a  100]( 
rating  being  achieved  only  by  an  efficient  DMU.  While  we  do  not  here  detail  the 
argument  from  economic  theory,  it  perhaps  suffices  to  say  that  all  of  welfare 
economics  rest  on  this  definition  of  efficiency,  l.e.  the  so-called  Pareto 
optimality  condition.  In  our  case  this  has  the  advantage  of  not  requiring 
us  to  assign  weights  on  an  a  priori  basis  to  the  various  educational  Inputs 
and  outputs.  Instead,  as  we  shall  see,  these  are  obtained  directly  from 
the  data  and  the  modes  we  shall  employ  In  an  objective  manner. 

In  our  example,  let  us  consider  four  hypothetical  schools  whose 
median  percentile  achievement  scores  for  beginning  and  end  of  year  are  as 
follows : 


School 

A 

B 

C 

D 

Posttest  Percentile 

60 

70 

65 

82 

Pretest  Percentile 

50 

60 

65 

80 

If  one  wants  to  evaluate  one  school’s  Pareto  efficiency  with  respect  to  the 
other  schools  in  the  set  for  the  single- Input-single-output  case,  one  does 
not  need  a  sophisticated  mathematical  model.  Common  sense  suffices: 

School  Output/Input  Output  =  f( Input) 


A 

d 

c 

D 


(60/50)  =  1.2 
(70/60)  =  l.lJ 
(65/65)  =  1 
(82/80)  =  1.025 


60  =  1.2(50) 

70  =  1. 1^(60) 
65  =  1(65) 

82  =  1.025(80) 


Clearly  school  A  gets  more  output  units  per  unit  of  input  than  the  other  schools. 
Note  that  results  here  are  descriptive.  What  Is  "causing"  the  different 
"production"  rates  Is  not  known. 


It  ahould  be  noted  that  re^aslon  of  poatteat  on  preteat  is  a 
frequently  used  deacriptlve  technique  and,  aa  la  ahown  below,  reaulta  from 
auch  analyala  for  the  aame  data  are  quite  different. 
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Model  for  Hypothetical  Schools: 


Fractional  Model 
Objective  (School  A): 

Max  h^  =  (60u/50v) 


Linear  Equivalent 


V  =  (6/5h. )u  for  fixed  h  where 
0<h^^1 


Constraints : 

School  A  Constraint 
(60u/50v)  ^  1 

School  B  Constraint 
(70u/60v)  ^  1 

School  C  Constraint 
(65u/65v)  S  1 

School  D  Constraint 
{82u/80v)  ^  1 


V  t  (6/5)u^»»u  ^  (5/6)v 

V  ^  (7/6)u<™»u  ^  (6/7)v 


V  ^  U  u  ^  V 


V  ^  (8.2/8)u  <  iL>  u  <  (8/8. 2)v 

u,v  >  0;  weights  employed  to  produce 
Pareto-efflclency  comparisons  with 
other  units  in  set. 


Note  that  school  A,  the  school  being  evaluated,  is  given  the  most  favorable 
weighting  that  the  constraints  allow;  that  is,  the  most  favorable  weighting 
possible,  given  that  all  schools  in  the  set  are  compared  on  the  same  basis. 
The  constraint  set  Insures  that  the  schools  are  evaluated  relative  to  each 
other  by  providing  a  maximum  value  for  the  objective  (right -hand- side 
constant  of  Inequality  constraints)  to  take  for  the  school  or  schools  doing 
best  relative  to  the  set  of  schools.  The  choice  of  this  constant  is 
arbitrary  in  that  whatever  the  maximum  is,  all  schools  will  be  compared  to 
that  same  value.  A  maximum  of  one  was  selected  for  modeling  and 
computational  convenience. 

Once  the  above  model  is  solved,  one  can  apply  the  Pareto  optimality 
criterion  to  determine  whether  or  not  school  A  is  Pareto  efficient  or 


inefficient : 


mry 
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If  <  1 ,  then  school  A  is  Pareto  Inefficient  because  a  school  or 
combination  of  schools  has  been  foxind  which  can  produce  the  same  amount  of 
output  with  less  input. 

If  =  1 ,  then  school  A  is  Pareto  efficient  in  that  no  such  combina¬ 
tion  described  above  could  be  found. 

In  Fifrure  2,  one  notes  that  iso-objective  line  for  maximum  h^  =  1  is 
coincident  with  school  A  constraint  and  feasible;  thus  school  A  is  Pareto 
efficient.  The  praph  in  Figure  2  can  also  be  used  to  evaluate  the  efficiency 
of  the  other  schools,  since  the  constraint  set  which  defines  the  feasible 
solution  space  remains  constant- -only  the  objective  changes  for  each  school. 
Further,  as  h  tends  toward  1  for  all  schools,  Iso-obJective  lines  become 
"flatter"  with  the  line  coincident  with  the  particular  school  constraint 
boundary  for  maximum  h  =  1  for  that  school.  Thus,  by  inspection  it  can  be 
seen  that  schools  B,  C,  and  D  are  not  efficient  in  that  their  respective 
cons tr  lint  boundaries  lie  outside  the  feasible  solution  space.  The  objective 
function  values  for  the  other  schools  can  be  interpreted  as  the  degree  of 
inefficiency  for  each  school  respectively;  in  other  words,  hg  =  .97, 
and  h^  =  .83  give  for  each  school  the  projxjrtlon  of  input  required  for  each 
school's  observed  output  if  these  schools  produced  as  efficiently  as  school 
A  and  the  unused  Inputs  (slacks)  for  schools  B,  C,  and  D  respectively  are 
2,  12,  and  11.  These  values  can  easily  be  verified  by  fixing  u  at  some 
convenient  value,  for  example  1 ,  and  solving  for  v  using  school  A's  objective 
function  since  h  is  known  to  be  1  at  all  points  on  this  line.  The  values 
for  u  and  v  can  then  be  used  in  the  objective  functions  for  the  other 
schools  to  solve  for  the  respective  efficiency  values  as  shown  on  the 
graph  in  Figure  2.  Slack  values  c£in  be  obtained  by  subtracting  h  times 
measured  input  from  the  measured  input. 


Graphic  Solution  to  Linear  I  ropranunlnp:  Moflel  t^ulvalent 


A  Constraint 

Constraint 

D  Constraint 
Constraint 


4 


A  mor«  I'omplpx  jrrnjihlc  lUuBtratlon  aiid  tiimplttx  taliUmviii 

Tor  H  two  Input-aliiKlt*  output  oxiunpl<>  can  bit  found  In  |  .ti  alonK  wltli  tho 

KunorallntKt  fora  of  the  nx.>dol  for  nultiplo  Inputs  and  outputs.  Thts  uhtdol 

Is  roprxHluoed  below  for  reader  oonvenienue. 

doneral  Model--Multlple  Inputs  and  outputs. 

a  m 

Objective:  MaxlmUe  c  ( ]iC  ''1*10^’  ^rO 

r=  1  I®  I 

designates  the  rth  output  measure  for  school  0-- 
the  school  being  evaluated.  There  are  s  output 
measures,  dnslgi\ates  tho  1th  Input  measure 

for  school  0.^  There  are  m  Input  measures,  u^.  and 
Vj  are  the  deslrtnl  weights. 

s  m 

Oonstralnta:  (  53  "-yn/S  J  *  ^ . » 

r=i  ^ 

Note  that  there  are  n  i\>nstralnts,  one  for  each  of 
the  J  ®  I . n  schools  In  the  set. 

'0,  '0.  x^j  '  0  for  all  1.  J.  and  r. 

Note  that  the  analysis  rtKjulres  a  model  for  each  school  In  the  set  as 
achiHils  must  be  evaluated  one  at  a  time.  Computational  efficiency  can  be 
galntHl  by  using  dual-simplex  methods  as  described  In  llOl,  pp.  M-I6.  The 
linear  programming  model  equivalent  to  tho  fractional  pivgrammlng  model 
given  alKwe  which  was  used  for  the  analysis  under  review  Is  f\illy  described 
In  1^1,  pp.  model  munbertHl  (3).  The  particular  linear  pivgraramlng 

model  used  was  readily  solvable  by  twisting  algvn’lthms  and  provided  (a) 
relative  off Iclency- inefficiency  values,  (b)  HuiTl'>i>‘*«  of  resources  under¬ 
utilised  by  Inefficient  schools  and  (c)  opin^rtunlty  costs  for  exhauattnl 
rttsources.  Ml'OS  (d)  was  used  to  solve  the  SS  ordinary  linear  programming 
models  which  werti  rt> formula! Ions  of  the  fractional  pn>graraming  models 


described  above. 
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AN  URBAN  SCHOOL  DISTRICT  APPLICATION 

An  urban  school  district  with  60,000  pupils  In  attendance  was 
chosen  for  the  application  because  a  recent  study  by  Jennings  [8)  provided 
measures  of  Input  from  school,  community  and  pupils  along  with  output 
measures  of  achievement.  Further,  a  close  working  relationship  with  the 
upper  level  administration  of  the  school  district  provided  a  means  for 
pursuing  administrative  response  to  the  Information  provided  by  the  analysis. 

The  55  elementary  schools  In  the  district  were  taken  as  the  decision 
making  units  (DHU's)  and  the  level  of  aggregation  of  all  data  was  the 
school  unit — outputs  expressed  as  medlsui  percentile  achievement  scores  for 
the  school  and  Inputs  expressed  as  school  totals,  ratios,  or  percents  as 
was  appropriate. 

As  Is  usual  for  such  schools,  large  amounts  of  federal  funds  were 
allocated  to  schools  on  the  basis  of  compensatory  education  efforts  for 
disadvantaged  pupils.  In  other  respects,  state  and  local  funds  were 
allocated  to  schools  on  a  uniform  basis  except  for  some  special  program 
funds  available  to  schools  for  subventing  attempts  to  improve  instruction 
on  an  R  &  D  basis. 

The  measures  obtained  for  the  analysis  were  categorized  as  follows: 
Output  Measures 

Four  outputs  measured  by  the  California  Achievement  Test  in  May, 


1977.* 

y^  median  percentile  reading  achievement  for  all  pupils  at  the 
school 

y^  median  percentile  reading  achievement  for  only  th„se  pupils 
In  attendance  at  the  school  for  a  full  year 


*Two  different  medians  were  computed  for  reading  and  mathematics 
achievement  in  order  to  have  an  Indication  of  the  "school  effect"  without  the 
Influence  of  pupil  mobility.  Thus  y^  and  y.  achlevonent  medians  are  based 
on  only  those  pupils  who  attended  no  other  school  during  the  year,  whereas 
y.  and  y,  scores  Included  pupils  who  attended  other  schools  before  treuisferrlng 
Into  the ^school. 
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median  percentile  mathematics  achievement  test  score  for  all 
students  In  attendance 

median  percentile  mathematics  achievement  test  score  for  only 
those  pupils  in  attendance  for  a  full  year. 

Input  Measxxres 

Pupil  inputs  measured  by  the  California  Achievement  Test  in  May,  1976. 
median  percentile  reading  achievement  for  all  pupils  at  the 
school 

median  percentile  reading  achievement  for  only  those  pupils 
in  attendance  at  the  school  for  a  full  year 
x^  median  percentile  mathematics  achievement  test  score  for  all 
students  in  attendance 

X|^  median  percentile  mathematics  achievement  test  score  for  only 
those  pupils  in  attendance  for  a  full  year. 

Proxy  measures  for  neighborhood  and  home  conditions  (obtained  from 

school  district  records) 

x^  percent  Mexican-Amerlcan  students 

Xg  percent  Black  students 

Xrj  percent  Anglo-American  students 

X0  percent  students  from  low  income  families 

x^  total  enrollment  In  schoool 

x^Q  percent  in  average  daily  attendance 

x^^  mobility  Index:  (total  enrollment  -  number  entered  late  or 
withdrawn ) /total  enrollment 

Proxy  measures  for  within  school  conditions  (obtained  from  school 

district  records) 

x^2  pupil- teacher  ratio 


^4 
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age  of  school  building 

per  pupil  expenditures  for  Instructional  supplies 
x^^  total  expendltiire  for  Instruction 
x^^  total  federal  funds  for  disadvantaged  children 
x^y  total  funds  for  all  major  special  programs  in  the  school 
x^g  percent  of  funds  for  Instruction  derived  from  local  sources 
School  organizational  climate  indicators  obtained  from  Organizational 
Climate  Description  Questionnaire  [^1;  a  high  score  on  each  dimension 
Indicates  the  following: 

disengagement — an  indicator  of  lack  of  cooperation  among  teachers 
hindrance — an  Indicator  of  the  extent  to  which  teachers  believe 
they  are  burdened  with  unnecessary  tasks 
esprit — an  Indicator  of  job  satisfaction 
Intimacy — an  indicator  of  how  much  social  interaction  exists 
among  teachers 

aloofness— an  indicator  of  Impersonal  treatment  of  teachers 
by  the  principal 

production  emphasis— an  Indicator  that  the  principal  is  task 
oriented  and  directive  In  supervision  of  teachers 
thrust — principal  motivates  teachers  by  personal  example  of 
work  orientation 

consideration — measure  of  the  principal's  friendliness  and 
cooperativeness  with  teachers 
Measures  of  classroom  instructional  processes  (obtained  from 
Individualization  of  Instruction  Inventory  (61;  higher  scores 
Indicate  greater  degree  of  individual  rather  than  group  oriented 
teaching  methods) 


*19 

*20 

*21 

*22 

*23 

*24 

*25 

*26 
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x^rj  intraclaas  grouping 

X2g  variety  of  teaching  materials 

^29  pupil  autonoiqy 
x^Q  differentiated  assignments 
tutoring 

x^g  total  Individualized  Instruction  Index 

Fifty-five  different  models  were  solved — one  for  each  school  euid 
the  comparative  efficiency  rating,  h^,  was  obtained  for  each  school.  An 
IIq  value  <  1  Indicates  that  the  associated  school  Is  Inefficient  with  respect 
to  the  measures  and  the  objective  process  employed  In  that  a  combination  of 
schools  have  been  found  which  C6Ui  produce  the  same  amount  of  output  with  less 
Input.  An  h^  value  =  1  means  that  no  such  combination  could  be  found  and 
thus  the  associated  school  Is  Pareto  efficient.  Slack  and  dual  variable 
values*  were  also  obtained  In  order  to  (a)  Inspect  the  extent  to  which 
Inputs  wore  being  under-utilized  by  DMU's  and  (b)  Inspect  the  opportunity 
costs  for  exhausted  resources.  The  obtained  h^  values  can  be  found  In 
Table  1  and  slack  values  for  Inefficient  schools  in  Table  2. 

Variable  Dependency  of  Results 

It  Is  clear  that  our  Intent  has  been  to  achieve  a  descriptive 
treatment,  but  the  question  still  remains — how  good  a  description  has  been 
obtained?  Since  the  rationale  has  been  very  straightforward,  with  no 
distributional  assumptions  about  the  data,  the  major  threats  to  descriptive 
validity  are  the  adequacy  of  the  variable  set  to  fully  describe  the  production 
system  and  the  sensitivity  of  the  classification  of  DMU's  to  the  Inclusion 
or  exclusion  of  Inputs  which  are  observed  to  correlate  with  outputs. 


*Note  that  for  efficient  schools  all  slack  variable  values  =  0. 


Table  1 

Comparative  Efficiency  Values  for  Schools 


Number 

DhU  Number  h^ 

DMII  N  timber 

1 

1.00 

20 

1.00 

3C 

2 

1.00 

?1 

1.00 

39 

3 

0.77^ 

22 

40 

4 

0.96* 

?3 

1.00 

41 

5 

1.00 

24 

1.00 

42 

6 

0.97* 

?5 

1.00 

43 

7 

0.96* 

26 

1.00 

4^1 

B 

1.00 

27 

0.95* 

9 

1.00 

28 

1.00 

46 

10 

0.97* 

29 

47 

11 

1.00 

30 

1.00 

12 

0.9?* 

31 

1 

l.OO 

49 

13 

3? 

50 

14 

1.00 

33 

1.00 

51 

15 

34 

1.00 

5? 

16 

1.00 

35 

53 

17 

36 

54 

18 

1.00 

37 

1.00 

55 

19 

1.00 

*  Denotes  an  inefficient  school 
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Table  2 

Slack  Variable  values  for  Schools  with  Low 
Comparative  Efficiency  Index 


VarlableA>I'’'’ 

3 

4 

6 

7 

10 

12 

27 

^0 

.77 

.96 

.97 

.96. 

.97 

.9? 

.95 

5.3 

5.2 

2.2 

1.8 

2.5 

5.9 

1.0 

<  . 

1.4 

2.5 

0.0 

0.0 

0.0 

4.4 

0.2 

’'3 

B.6 

0.0 

7.1 

11.5 

4.6 

4.6 

0.0 

X4 

7.1 

3.3 

6.4 

15.2 

2.6 

0.1 

0.0 

0.0 

10.  n 

10.3 

0.0 

0.0 

21.2 

23.7 

’'6 

29.3 

21.3 

6.6 

25.7 

70.6 

0.0 

0.0 

’‘7 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

0.0 

y^,o 

0.0 

0.0 

25.6 

58.5 

30.8 

12.4 

2.4 

1.6 

0.0 

0.0 

0.0 

3.7 

6.0 

*10 

21.6 

26.4 

12.7 

17.6 

52,4 

15.3 

20.5 

*11 

23.2 

7.8 

0.0 

17.3 

69.3 

0.0 

6 .6 

*12 

7.2 

8.2 

2.5 

3.2 

13.6 

2.7 

6.7 

*13 

28.8 

7.7 

17.5 

5.1 

35.1 

21.9 

0.0 

*14 

0.0 

2.0 

0.1 

2.5 

3.2 

0.9 

1.2 

/15 

2.8 

47.8 

14.6 

17.8 

36.2 

261.5 

376.8 

16 

42.5 

0.2 

0.8 

74.1 

0.0 

45.2 

0.9 

^’'17 

72.7 

0.0 

17.0 

91.0 

19.2 

75.8 

0.0 

*1B 

21.4 

28.0 

l'--.8 

5.9 

57.8 

12.5 

22.1 

’'19 

27.4 

32 .6 

3%5 

37.9 

56.9 

9.4 

37.5 

*20 

12.8 

28.2 

19.2 

"0.0 

'i4.6 

^'.3 

13.6 

*21 

41.2 

31.2 

35.2 

36.2 

53.4 

24.5 

17.4 

*22 

30.6 

27.1 

26.3 

32.6 

50.9 

6 .6 

17.7 

*23 

31.3 

19.6 

22.2 

24.9 

43.2 

0.0 

12.5 

*24 

3'^.  3 

28.3 

34.2 

25.6 

56.7 

12.5 

10.9 

39.4 

22.2 

30.3 

4.7 

50.6 

33.8 

0.0 

*26 

41.6 

27.3 

32.7 

14.5 

56.2 

33.5 

11.7 

*27 

27.7 

35.0 

18.1 

2.3 

47.6 

8.7 

19.0 

*28 

18.4 

39.2 

12.7 

10.2 

3S.0 

9.3 

15.7 

*29 

13.9 

27.1 

15.7 

7.6 

35.4 

7.8 

12.4 

<-  ••  * 

30.2 

19.2 

0.0 

40.7 

15.7 

9.7 

*31 

:5.4 

34.5 

22.0 

27.5 

40.1 

:’0.9 

3.3 

*32 

24.3 

33.0 

17.9 

10.1 

40.0 

12.5 

12.1 

0 

0 

1 

c 

^unlt 

-  $1000 
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Our  procedure  for  this  rudimentary  sensitivity  analysis  was  as 

follows: 

a.  A  least  squares  regression  of  each  output  variable  on  the  set 
of  Input  variables  to  establish  whether  a  predictive  relationship  existed 
In  the  data  set. 

b.  Given  an  affirmative  result  In  (a)  above,  a  test  of  units  found 
to  bo  in  the  less  productive  set  to  see  If  their  classification  changed  with 
a  reduced  set  of  Input  variables — excluding  those  having  high  correlations 
with  outputs. 

c.  A  discriminant  analysis  to  cross-valldate  the  classification  and 
to  provide  management  Information  on  the  Input-output  dependencies  (In  the 
data.  If  not  In  the  field). 

In  test  one,  the  regression  models  showed  a  high  relatlonslp  of 
2 

Inputs  to  outputs  (R  >  0.92)  for  each  output  used  one  at  a  time  as  the 
criterion  variable  and  the  complete  set  of  input  variables  as  predictors. 

The  standard  error  of  estimate  was  <  6.0.  As  might  be  guessed,  achievement 
Input  (pretest)  scores  were  the  best  predictors  of  output  achievement, 
thus  In  step  6  we  resolved  the  linear  programming  models  for  all  ten  schools 
In  administrative  area  1  of  the  district  with  entering  achievement  deleted 
from  the  Input  set.  Five  of  these  ten  schools  had  initially  been  classified 
as  efficient  (h^  =  1)  and  five  were  Inefficient  (hp  <  1).  The  changes 
resulting  from  the  reanalysls  are  shown  In  Table  3. 
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.'able  T 

Oop.j'ra'lr.ori  of  D  a'er.ullr  for  Are.a  1 
'Ichooln,  Oont.rollln  '  for  >iiterlnf  Achlcver.ent 


t  Initial  Valuer 
(r.ll  Irpnt  variables) 

1  1.00 

0  1.00 

■»*  0.77 

A"  O.Of 

I.OO 

^97 

0.<V 
1.00 
1 . 00 


10* 


fl  I'.’T 


riedueefl  hp  Values 
(entorinr,  achievement  deleted) 

1.00 

1.00 

0.7? 

1  .OC 
O.Oe. 

a  a-' 

•  ^ 

l.CC 
1  .00 
0.01 


'b.rinl'^re 

O.vOO 

0.00 

0.00 

0.0? 

0.00 

0.0-^ 

0.11 

O.OG 

r.oo 

'.0' 


* Int>fric!  i-nt  rciioolr. 


We  see  In  Table  3  that  dropping  the  Input  measures  most  highly 
correlated  with  output  did  not  change  the  classification  of  school  units, 
but  did  result  in  a  shrinkage  in  h^  for  leas  productive  schools.  This  is 
the  result  one  would  expect  if  the  analysis  is  not  highly  sensitive  to 
changes  In  the  Input  variable  set  (no  change  in  classification).  Shrinkage 
in  h^  values  makes  schools  appear  to  be  only  sll^tly  more  inefficient  when 
a  major  pupil  input  was  not  considered. 

The  explanation  for  the  robustness  of  classification  offered  here 
is  that  the  input  variable  set  is  broad  enough  to  Include  correlates  of 
the  input  variable  deleted,  and  the  effects  of  the  missing  variable  are 
still  accounted  for. 

It  would  seem,  then,  that  If  no  correlates  of  the  excluded  measure 
were  In  the  model,  a  change  In  classification  would  likely  occur  if  the 
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excluded  Input  la  an  inportant  predictor  of  output.  A  test  of  this 
speculation  was  conducted  by  excluding  all  six  variables  correlated 
(r  >  .50)  with  achievement  measures: 

a.  entering  achievement 

b.  percent  minority  group  enrollment 

c.  percent  students  from  low  Income  families 

d.  percent  students  In  attendance 

e.  total  federal  funds  for  disadvantaged  children 

f.  total  funds  for  major  special  programs 

The  model  for  school  #1,  classified  as  efficient,  was  reanalyzed 
with  the  six  achievement-correlated  variables  listed  above  deleted  from 
the  input  set.  The  resulting  value  of  h^  was  ,999976  which  might  indicate 
either  round-off  error  or  that  school  ^f^  would  be  reclassified  as  in¬ 
efficient.  In  any  event,  the  change  in  h^  was  quite  small. 

The  conclusion  drawn  from  this  result  was  that  DEA  should  be 
preceded  by  a  preliminary  analysis  to  ascertain  that  Input  measures  are 
correlated  with  outputs  and  that  measures  representing  all  Important 
aspects  of  the  school  production  system  are  Included  in  the  model. 

DISCRIMINANT  ANALYSIS:  EFFICIENT  VS  INEFFICIENT  SETS 

A  unique  contribution  of  the  Data  Envelopment  Analysis  is  that  It 
provides  a  way  to  classify  decision  making  units  Into  efficient  and  In¬ 
efficient  sets  when  no  a  priori  basis  exists  for  such  partitioning.  That 
Is,  beginning  with  a  set  of  measures  on  units,  those  that  are  equally 
efficient  are  distinguished  from  the  others.  This  Is  converse  to  discriminant 
analysis  which  begins  with  a  known  classification  of  units  into  groups  and 
accounts  for  the  grouping  by  linear  combinations  of  measures  obtained  on 
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the  units.  In  DEA,  no  readily  available  means  exists  for  Judging  the  relative 
success  of  the  partitioning  in  terms  of  the  discriminating  power  of  the 
measures  employed  to  achieve  the  classification  Into  efficient  and  In¬ 
efficient  sets.  In  the  use  of  discriminant  analysis,  however,  one  has  both 
statistical  Indicators  of  discriminating  power  and  a  heuristic  check  by 
using  the  linear  discriminant  function  to  replicate  the  classification. 

Thus,  It  would  seem  that  the  two  procedures  are  complementary  with 
discriminant  analysis  adding  a  cross-validation  of  outcomes.  Accordingly, 
a  discriminant  analysis  was  performed  and  Interpreted  as  follows. 

Procedure 

Using  SPSS,  version  7.0,  subprogram  DISCRIMINANT  (12],  a  two-group 
analysis  was  performed  for  efficient  (n^  =  k8)  and  inefficient  (n^  =  7) 
schools.  Classification  was  adjusted  for  unequal  size  of  groups.  The 
same  Input  variables  defined  earlier  to  x^^)  were  used  as  discriminating 
variables  with  a  single  exception.  Percent  of  Anglo  enrollment  was  deleted 
to  eliminate  the  linear  dependency  of  the  variable  with  percent  Black  and 
Hex Ican-Amerlcan . 

A  discriminant  function  with  considerable  discriminating  power  was 
obtained:  canonical  correlation  =  0.88  and  Wilk’s  Lambda  =  .219. 

One  hundred  percent  success  of  classification  of  DMU's  into  the 
correct  set  was  found.  While  there  was  likely  some  overflttlng  because 
the  same  variables  were  used  In  DEA  to  create  the  subsets  and  In  discriminant 
analysis  to  classify  DMU’s  Into  the  subsets,  the  results  indicate  a  high 
discriminating  power  even  allowing  for  some  shrinkage. 

To  Illustrate  the  marked  separation  achieved  by  the  discriminant 
function,  the  discriminant  scores  for  the  ten  schools  In  a  single  area 
of  the  district  (same  schools  as  in  Table  3)  are  given  in  Table  4. 
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?able  h 

Dlr.crlminant  .Jcoro",  for  i^fflcient 
anf’.  Inefficient  .‘ichoolc  In  Area  1 

DiscRilfiwr  snoR:^r! 

nmcii'Tri'  Din."r,  iNEFFiciar''  nn;'2 


n!:u 
1 
2 

3 

4 

5 

6 

r» 

( 

F 

9 

10 

Gentroi'l 


0.86 

0.51 


0.16 


-1.03 

-0.54 

0.914 


-7.3? 

-6.7? 

-6.37 

-5.?'> 


-7.10 

.'57 


DISCUSSION 

We  have  shovm  that  we  can  Identify  Individual  school  units  that 
are  less  efficient  than  other  comparable  units  in  terms  of  measured  achieve¬ 
ment  scores  relative  to  input  factors  representing  enterlniT  achievement, 
school  neighborhood  characteristics,  expenditures  for  instruction,  type  of 
instruction,  and  attributes  of  faculty  and  principal.  It  is  beyond  the 
scope  of  the  present  analysis  to  determine  administrative  reallocation  of 
resources  in  order  to  achieve  greater  overall  efficiency.  What  we  can  do, 
however,  is  a  significant  improvement  in  presently  available  management 
information  in  the  administration  of  schools — we  can  identify  inefficient 
units  and  show  what  school  input  factors  contribute  to  their  less  efficient 
status  by  being  imderutilized.  The  administrative  response  to  such 
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Information  should  not  necessarily  be  to  reduce  the  slack  resource.  In 
some  cases  the  resource  Is  mandated  by  law  and  is  allocated  on  a  formula 
basis  and  even  when  local  leeway-  is  present,  the  absence  of  causal  evidence 
would  axiggeat  the  need  for  a  cautious  management  response. 

What  is  suggested  then,  is  a  three  step  procedure  for  the  use  ol 
DEA  results  in  school  districts:  (a)  Identification  of  inefficient  units 
by  top-level  administration  and  reporting  of  results  to  individual  schools, 
(b)  consideration  of  slack  variable  values  by  school  unit  administrators 
and  their  interpretation  In  terms  of  changes  in  the  school  unit  operation 
targeted  to  the  Improvement  of  efficiency,  and  (c)  reanalysis  at  the  time 
of  the  next  achievement  testing  to  determine  if  adjustments  indicate 
Improvonents . 

Let  us  illustrate  these  steps  by  a  discussion  of  a  possible  scenario 
for  school  #3,  the  most  inefficient  unit. 

Step  (a) — As  was  shown  in  Table  1,  the  h^  value  for  school  3  is  0.77. 
This  may  be  interpreted  to  indicate  that,  compared  to  other  units,  school 
13  should  be  able  to  achieve  Its  productivity  rate  by  using  only  77%  of 
the  total  Inputs  currently  observed  in  that  school.  If  achievement  is 
low  in  school  3,  this  would  suggest  strong  efforts  are  needed  to  improve 
achievement  without  additional  resource  allocations.  If  achievement  is 
high  relative  to  other  schools,  top  administration  might  consider  realloca¬ 
tion  of  some  resources  ^  other  low  achieving  but  efficient  units.  School 
#3  actually  had  68%  in  reading  scores  and  76%  in  mathematics  scores.  These 
are  in  the  high  average  range  of  productivity.  Looking  further,  however, 
we  see  that  the  school  had  higher  reading  percentile  achievement  scores 
the  previous  year  (75tlle).  Thus  the  Inefficient  status  of  the  school 
would  seem  to  be  partly  attributable  to  lowered  achievement  (output)  relative 
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to  previous  achievement  (Input).  Other  factors,  of  course,  are  involved 
and  we  will  pass  the  results  on  to  the  principal  and  faculty  of  school 
#3  for  their  on-the-scene  interpretation. 

The  implication  of  the  above,  however,  is  that  the  school  needs 
to  regain  its  previous  achievement  level  without  additional  school 
district  resources  under  the  threat  that,  since  its  achievement  is  not 
among  the  lowest  in  the  district,  some  reallocatable  resources  may  be 
removed.  Efficiency  ratios  can  also  be  brought  into  balance  by  reducing 
underutilized  Inputs. 

Step  (b) — The  staff  of  DMU  ^3  must  now  consider  the  Indicators  of 
how  they  differ  from  more  efficient  units.  Table  5  provides  a  profile 
of  variables  that  are  illustrative  of  the  Inputs  that  the  staff  of  the 
DMU  might  wish  to  consider.  First,  the  mean  values  for  the  schools  in 
the  efficient  and  inefficient  sets  are  shown,  followed  by  the  observed 
value  for  the  DMU.  Finally,  the  slack  variable  value  from  the  DEA  results 
for  school  #3  indicates  "excess"  resources.  That  is,  if  the  production 
rate  of  school  #3  were  the  same  as  schools  with  the  greatest  production 
rate,  then  input  resource  i  could  be  reduced  by  as  much  as  s^  without 
any  associated  reduction  in  y^  (output). 
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The  staff  ml(;ht  consider,  for  eocample,  that  two  of  the  factors, 
Dlsenfa^eoent  and  Hindrance,  are  negative  attributes  which  they  have  in 
greater  measure  than  other  schools.  Accordingly,  they  might  propose  to 
reduce  the  "excess"  units  by  greater  commitment  to  Instructional  efforts 
(engaging  In  the  central  task  of  the  school)  and  by  reiluclng  the  amount 
of  hindrance  caused  by  burdensome  non- teaching  routines.  Supposedly, 
this  gain  In  staff  energy  and  attention  might  be  employed  to  Increase 
the  use  of  those  underutilised  Instructional  methitds  detected  In  mort' 
efficient  schools.  Thus,  Increased  pupil  autonomy  and  Individualised 
instruction  might  be  employed  more  effectively  In  an  effort  to  Increase 


achievement. 


If  aome  analysis  such  as  the  one  suggested  were  made  by  the  staff 
of  school  l3i  then  they  would  be  able  to  propose  a  modified  operating 
plan  for  the  subsequent  year.  This,  along  with  district-level  modifications, 
would  become  the  goal-setting  vehicle  for  the  DMU. 

Step  (c) — After  a  year  of  operation  under  the  modified  pleui, 
measures  would  again  be  obtained  and  DBA  computed  for  all  schools.  If  the 
efficiency- Increasing  steps  have  been  effective,  it  would  bo  detectable 
by  an  increase  in  hg  or  perhaps,  in  the  best  outcome,  the  DMU  would  be 
classed  as  efficient.  Of  course,  in  the  unlikely  event  that  all  schools 
in  the  district  improved  their  effectiveness,  no  change  or  even  a  drop  in 
h^  could  be  observed,  since  it  is  a  comparative  measure.  However,  in 
this  case,  an  increase  in  outputs  for  all  the  schools  should  be  observed. 

In  this  fashion,  the  analysis  provides  the  basis  for  needs  identifica¬ 
tion,  program  planning  and  evaluation.  These  are  all  needed  management 
tools  for  school  district  administration. 

SUMMARY 

Management  of  schools  has  been  hindered  by  lack  of  appropriate 
analytical  tools.  A  technique  called  Data  Envelopment  Analysis  (DEA)  has 
been  employed  to  measure  the  productivity  of  individual  schools  in  an 
urban  school  district  and  to  identify  those  that  are  less  efficient  than 
others  with  respect  to  the  Parato-Koopmans  Optimality  Criterion. 

The  results  of  DEA  were  cross-validated  by  means  of  discriminant 
analysis  and  the  results  were  confirmed.  Finally,  a  discussion  of 
results  presented  the  outline  of  a  procedure  for  using  DEA  results  as 
management  information  for  the  improved  efficiency  of  schools. 
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